Papers with Learning high-quality sentence representations
A Contrastive Framework for Learning Sentence Representations from Pairwise and Triple-wise Perspective in Angular Space (2022.acl-long)
Copied to clipboard
| Challenge: | Existing methods for learning sentence representations focus on constitution of positive and negative representation pairs and do not focus on training objective. |
| Approach: | They propose a new method to learn sentence representations using BERT-like pre-trained models . they use a pairwise discriminating power and a model to model the entailment relation of triplet sentences . |
| Outcome: | The proposed method outperforms the previous state-of-the-art on diverse sentence related tasks. |
ConSERT: A Contrastive Framework for Self-Supervised Sentence Representation Transfer (2021.acl-long)
Copied to clipboard
| Challenge: | Existing BERT-based pre-trained language models achieve high performance on many downstream tasks, but native derived sentence representations are collapsed and thus poor performance on semantic textual similarity (STS) tasks. |
| Approach: | They propose a framework for self-supervised Sentence Representation Transfer that adopts contrastive learning to fine-tune BERT in an unsupervised way. |
| Outcome: | The proposed framework improves on the BERT-derived representations by 8% on STS datasets and shows robustness in data scarcity scenarios. |